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Parent/Student Differences in Evaluation 2 
Abstract 

The purpose of this study was to determine if there were differences in evaluation of an 
educational interactive video program between high-school students and their parents over a 4- 
year time span. High school students participating in the interactive video program were survey 
during 5 semesters. Survey instruments were also sent to parents via the students. There were 
1024 student respondents and 481 parent respondents. Constructs for each group were 
determined using exploratory factor analysis. The mean vector of similar constructs for each 
status group were contrasted using multivariate analysis of variance (MANOVA) with semester 
and site as additional factors. In addition, similar questions for each status group that would not 
included in a construct were also contrasted by MANOVA. Because homogeneity of covariance 
matrices was not achieved in the initial sample, a stratified random sample of 150 was 
selected from each site and status group and the analysis repeated. Results indicate there are no 
differences in evaluation of the interactive video program between parents and students. These 
results are discussed. 
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High School Student - High School Parent: 

Are there Differences in Evaluation of an Educational Interactive Video Program? 

Increased use of interactive video (ITV) for distance learning students has made classes 
previously inaccessible to rural high school students available locally (Monaghan, 1996). 

Offering college courses at remote sites by interactive video permits students to take the class 
without a lengthy drive. In some localities homebound students may now partake of classroom 
interaction by interactive video. Thus, we may perceive interactive video as a means of providing 
equal educational opportunities to all students. There are, however, questions concerning this 
program. 

Although interactive video technology has advanced rapidly in recent years, there is 
increasing evidence that no one technology works in every application (Linking, 1989). 

In addition, the technology utilized by interactive video requires a different preparation for 
teaching than traditional methods (Knapczyk, 1993). While the expenditure required for 
interactive video may be less than providing on site teachers (Morgan, 1994; Villarroel, 1988) or 
may provide access to equal educational opportunities in rural areas, there is an increasing need 
for research into and evaluation of the effectiveness of interactive video programs. Evaluation of 
programs, however can be costly. Although convergent information from different sources 
provides evidence of validity, if information collected is redundant, the expense of collecting and 
analyzing data could be used more beneficially. 

The primary purpose of this study was to determine if there were differences in the 
evaluation of an educational interactive video program between participating high school students 
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and their parents. Yearly results have been utilized to provide improvements to the program. If, 
however, both parents and students provide the same information, data collection can be limited 
to only one group. 

Literature Review 

The basic criterion for distance education is distance between the teacher and the student. 
Distance education is not new. This technique was begun in the nineteenth century with 
correspondence education (Klesius, Homan, & Thompson, 1997). It has, however, changed from 
the correspondence delivery method, through radio methods, to today’s computer and interactive 
video techniques. 

Currently, distance education has been used for high school students as an alternative 
method to earn credentials in the General Education Development (GED) program, to obtain 
college credits (Green, 1996), or in attempts to revitalize curricular programs (Fucci & Hueston, 
1997). Some universities have developed dual degree partnerships with interested businesses to 
provide on-site, on-demand graduate programs (Haynes & Pouraghabagher, 1997). And, some 
universities have developed programs to deliver education to rural areas or cultural groups 
(Monaghan, 1996). 

Prior researchers in distance education have investigated student satisfaction, 
communication techniques, teaching behavior, and change fostered (Moore & Thompson, 1990). 
Because distance education places students in the situation in which there may be no interaction 
or association with other students or the teacher, system requirements must be sound 
(Gunawardena, 1988; as cited in Dillon, Gunawardena, & Parker, 1992). Carter (1997) found that 
audio was the most important element of interactive education, followed by lighting. 
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In addition, the importance of the role of the teacher or facilitator has been stressed by 
several researchers (Garrison & Baynton, 1987 as cited in Dillon, Gunawardena, & Parker, 1992) 
and interaction with the instructor has been central to the success of a distance education 
program. When a distance education program has active support, some researchers have found no 
differences in program rating between home and remote sites. Thyer, Polk, and Gaudin (1997), 
however, reported that live instruction was rated significantly higher at a college campus than 
distance learning. They add that distance learning has not yet demonstrated comparable outcomes 
in terms of student learning. 

Developing courses for distance education can be extremely expensive. A properly 
equipped distance education classroom may cost more than $75,000 (Swift & Wilson, 1997). In 
addition, there are other costs including instructors training. Although the use of distance 
education provides the obvious advantage to take otherwise unavailable classes, as the role of 
distance learning expands, it is essential that the problems unique to this format be examined 
(Wilson, Litle, Coleman, & Gallagher, 1997/98). How do programs change over time? What do 
students perceive as advantages and disadvantages of the distance education program? What do 
parents perceive as advantages and disadvantages? Should information be collected fi-om all 
stakeholders? 

The current study sought to answer questions concerning information collection. Of prime 
concern was to determine if there were differences in evaluation of the ITV program between 
parents and students. In addition to this omnibus question, however, we sought to determine if 
site or semester interacted with parent-student status providing differences in response. 
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Method 

Subjects 

High school students enrolled in an interactive video class at a facility in a rural 
Applachian area during the Spring semesters of 1995, 1996, 1997, and 1998 and the Fall semester 
1997 were surveyed. Surveys were administered during the regularly scheduled class time by the 
class instructor or remote facilitator. In addition, similar questionnaires were sent to the parents 
via the students. 

Measurement 

The original survey instrument consisted of demographic information (school, gender, 
grade, etc.), some 5-point Likert style questions, and three open-ended questions concerning 
strengths, weaknesses, and suggested improvements for the interactive video program. The Likert 
style questions were used for this analysis. 

Responses to the Likert-style questions ranged from 1 (Strongly agree) to 5 (Strongly 
disagree) and a non-applicable category. Some Likert-type questions were negatively stated in the 
questionnaire. Negatively stated questions (negative toward the ITV program) were reverse 
coded for this analysis. The non-applicable category was treated as missing data. 

Analysis 

Exploratory factor analysis for each status group was used to determine basic constructs 
to be investigated. Because student questionnaires contained 35 Likert-type questions and parent 
questionnaires contained 30 questions, there were different numbers of questions loading on each 
construct for each group. Coefficient alpha was determined for each construct for each group. If 
coefficient alpha for a construct was less than 0.6, the construct was considered to be unrefeble 
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and was not included as a construct. 

Student and parent constructs were then compared visually to determine if they contained 
similar questions. If questions on a construct did not match similar questions from the other 
group, that construct was eliminated. Six student constructs corresponded to parent constructs. A 
mean score for each of these constructs was determined for each respondent. Student questions 
not included in a construct corresponding to a parent construct but had a matching question on 
the parent questionnaire were analyzed separately. 

Constructs were analyzed using multivariate analysis of variance (MANOVA). When 
multivariate significance (p< .05) was detected, the mean vector was fiirther analyzed by 
considering the proportion of variance explained. If status, the interaction of status and site or the 
interaction of status and semester contributed to multivariate significance and explained over 5% 
of the variance in the mean vector, each construct was examined univariately. If a construct was 
significantly different (p < .05) by status or either of the status interactions and 5% of the 
difference in the construct was explained by status or the status interactions, each question 
loading on this construct was analyzed as an individual dependent variable. In addition, questions 
not included in a construct but having a matching question in the other group were analyzed by 
MANOVA. 

Results and Discussion 

There were 1505 respondents (students = 1024; parents=481) included in this comparison 
for the four survey years (5 semesters) in this analysis. Exploratory factor analysis yielded 6 in- 
common constructs for the parent and student questionnaires. Each construct had a reliability 
greater than 0.6 for their group (see Table 1). 
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Insert Table 1 About Here 



A mean was determined for each respondent for each construct (see Table 2). In addition, 
5 questions were similar for the respondent groups. Although 3 parent questions and 4 student 
questions appeared to be similar, the wording of the questions prevented their comparison (see 
Table 3). 



Insert Tables 2 and 3 About Here 



Initial analysis of the 6 in-common constructs resulted in a statistically significant 
difference (F=3.82, df=6, 1436, p<.01) in mean vector between students and parents. Less than 
2% (T)^ =.016) of the variance in mean vectors, however, was accounted for by differences in 
status group. In addition, equality of covariance matrices was not achieved. Consequently, a 
stratified random sample of 600 respondents (300 students, 300 parents) equally distributed 
between home and remote sites was selected. The analysis was again conducted yielding a non- 
significant F ratio of 1 .28 (df=6, 550, p>.05). Status accounted for 1 .4% of the variance in this 
analysis. In both instances less than 2% of the variance in mean vector was explained . 

Because equality of covariance matrices was not achieved initially, all further analyses 
were conducted on the full sample as well as the reduced number. Although it would have been 
preferable to have conducted the analysis using a MANOVA which included status, site, and 
semester concurrently, this was not done due to the disparity in cell sizes. Consequently, multiple 
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analyses were conducted. Construct means were first contrasted by site and status and then 
contrasted by semester and status. Then the 5 in-common questions were contrasted by site and 
status. This was followed by a contrast of these 5 questions by semester and status. 

Yearly analyses had yielded statistically significant differences between sites for both 
status groups. It was not surprising, therefore, to find significant differences by site (F=45.1, df 
=6, 550, p<.01). Site accoimted 33% of the variance in the constructs. The primary concern with 
this analysis, however, was to determine if there were interaction effects between site and status 
group. There was also a significant interaction effect (F=2.43, df= 6, 550, p< .03). Although 
statistically significant, the interaction between site and status accoimted less than 3% of the 
variance in constructs (see Table 4). 



Insert Table 4 About Here 



Constructs were also contrasted by status and semester using the full and the reduced 
sample. Statistically significant main effects of status, semester, and the interaction of status and 
semester were detected. None of these effects accounted for 2% of the variance in constructs (see 
Table 4). 

The analysis was then conducted to determine if there were differences between the 5 in- 
common questions which could not be included in similar constructs. There were statistically 
significant differences in the reduced sample mean vector (F=2.3, df=5, 528, p<.05) between 
students and parents explaining 2.1% of the variance. Similar results were obtained fi"om the full 
sample (F=4.86, df= 5, 1351, p < .01; T|^=.018) and when status was contrasted with semester 
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(see Table 4). There was no statistically significant interaction effect between status and site 
(variance accounted for < 2%) or status and semester (variance accounted for <1%). 

Two groups of questions were not contrasted. Some parent questions were worded in a 
different manner than students. For example, parents received the statement, ‘Study is the same 
for ITV as for traditional classes’. Students received statements such as, ‘I study for ITV’ and ‘I 
study for traditional classes’. In this instance, the parent questions were questionable. 

Conclusion 

Although statistically significant differences were detected between parents and students, 
status did not account for 5% of the variance. This suggests that this portion of the questionnaire 
is providing similar information fi-om parents and students. In addition, the statistically significant 
interaction effects between status and site and between status and semester did not account for 
5% of the variance. Thus, it was concluded that collecting both parent and student information is 
redundant. 
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Table 1 



Reliability and Questions included for Factors bv Status 





Question Included 
Parent Student 


Reliability (Coef Aloha 
Parent Student 


ITV Evaluation 






0.77 


0.89 


ITV Good Addition Currie 


/ 


/ 






Par ITV good addition 




/ 






Hestitate Take Another ITV 


/ 


/ 






Choice - ITV Class 


/ 


/ 






ITV Good Way Offer Class 


/ 


/ 






Take Coll course on ITV 


/ 


/ 






Better Listener 


/ 


/ 






Teacher Interaction 






0.8 


0.83 


ITV teacher knows me 


/ 


/ 






Can Hear Teacher 


/ 


/ 






Can Ask Quest 


/ 


/ 






Teacher hears me 


/ 


/ 






Talk to Teach as needed 


/ 


/ 






Materials Suooort 






0.64 


0.68 


Class materials timely 


/ 


/ 






Returned Work 


/ 


/ 






Teacher Remote as Needed 


/ 








Cheatina 






0.65 


0.67 


Cheating Trad Classes 




/ 






Easier Cheat Remote 


/ 


/ 






Easier Cheat Home 




/ 






Poor Behav ITV 


/ 


/ 






Obs Cheating ITV 


/ 


/ 






Environment 






0.6 


0.65 


Amt Desk Space 


/ 


/ 






Clear sight TV 


/ 


/ 






Attractive Classroom 




/ 






See materials on syst 


/ 


/ 






Audio 






0.79 


0.8 


Hear Students other sites 


/ 


/ 






Hear Quest other Sites 


/ 


/ 
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Table 2 



Factor and Question Means by Status 







Factor Means 


Question Means 


Factor/ 

Category 


Question 


Parent Student 


Parent Student 



ITV Evaluation 


ITV Good Addition Currie 


2.53 


2.5 


2.23 


2.27 




Par ITV good addition 








2.5 




Hestitate Take Another ITV 






2.6 


2.54 




Choice - ITV Class 






3.12 


2.97 




ITV Good Way Offer Class 






2.08 


2.1 




Take Coll course on ITV 






2.45 


2.47 




Better Listener 






2.63 


2.75 


Teacher Interaction 


ITV teacher knows me 


1.9 


1.95 


3.2 


2.2 




Can Hear Teacher 






2.61 


2.05 




Can Ask Quest 






2.23 


1.93 




Teacher hears me 






2.66 


2.19 




Talk to Teach as needed 






1.72 


1.95 


Materials Support 


Class materials timely 


2.07 


1.99 


1.79 


2.02 




Returned Work 






2.09 


2.01 




Teacher Remote as Needed 






3.35 




Cheating 


Cheating Trad Classes 


2.81 


2.75 




3.32 




Easier Cheat Remote 






3.03 


3.24 




Easier Cheat Home 








2.29 




Poor Behav ITV 






2.96 


2.6 




Obs Cheating ITV 






2.64 


2.62 


Environment 


Amt Desk Space 


1.75 


1.78 


1.81 


1.61 




Clear sight TV 






1.8 


1.53 




Attractive Classroom 








2.45 




See materials on syst 






1.65 


1.87 


Audio 


Hear Students other sites 


2.18 


2.33 


2.18 


2.27 




Hear Quest other Sites 






2.2 


2.37 
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Table 3 



Question Means bv Status (questions not loading on a factor^ 







Question Means 


Category 


Question 


Parent 


Student 


Matched Questions 


Limit ITV affect Grade 


2.89 


2.78 




Know Stud Other Schl 


2.67 


2.67 




Meet Other Schl Stu more often 


2.42 


2.23 




Most Talk by Homesite 


3.05 


3.25 




ITV Courses Difficult 


2.93 


2.79 




Trad Courses Difficult 




2.84 


Unmatched Questions 


Tchr attn same home and remote 
Tchr attn home site 


2.5 


3.37 




Tchr attn remote site 




2.37 




Study for ITV 




2.65 




Study for Trad Class 
More Study ITV 


2.8 


2.43 




Study Same ITV 


2.34 
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Table 4 

Tests of Statistical Significance and Variance Explained 







Eaual Size Grouns 


Total Samole 


Model 


Effect 


F“ Eta^ 


F“ Eta^ 



Constructs 



1 


Status 


1.28 


.014 


3.98** 


.016 




Site 


45.10** 


.330 


94.46** 


.284 




Status/Site 


2.44 


.026 


3.22** 


.013 


2 


Status 


1.59 


.017 


3.76** 


.016 




Semester 


1.77* 


.019 


3.33** 


.014 




Status/Semester 


1.51 


.016 


2.13** 


.009 


Questions 


3 


Status 


2.30* 


.021 


4.86** 


.018 




Site 


3.42** 


.031 


6.55** 


.024 




Status/Site 


2.13 


.020 


2.11 


.008 


4 


Status 


2.03 


.019 


3.08** 


.011 




Semester 


1.90** 


.018 


2.99** 


.011 




Status/Semester 


.79 


.007 


.89 


.003 



Note . “As measured by Wilks’ Lambda. *p< .05. **p< .01 . 
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